Computing a Classic Index for Finite-Horizon Bandits

Computing a Classic Index for Finite-Horizon Bandits

T paper considers the efficient exact computation of the counterpart of the Gittins index for a finitehorizon discrete-state bandit, which measures for each initial state the average productivity, given by the maximum ratio of expected total discounted reward earned to expected total discounted time expended that can be achieved through a number of successive plays stopping by the given horizon...

Regret Analysis of the Finite-Horizon Gittins Index Strategy for Multi-Armed Bandits

I prove near-optimal frequentist regret guarantees for the finite-horizon Gittins index strategy for multi-armed bandits with Gaussian noise and prior. Along the way I derive finite-time bounds on the Gittins index that are asymptotically exact and may be of independent interest. I also discuss computational issues and present experimental results suggesting that a particular version of the Git...

Computing an index policy for multiarmed bandits with deadlines

This paper introduces the multiarmed bandit problem with deadlines, which concerns the dynamic selection of a live project to engage out of a portfolio of Markovian bandit projects expiring after given deadlines, to maximize the expected total discounted or undiscounted reward earned. Although the problem is computationally intractable, a natural heuristic policy is obtained by attaching to eac...

A SIMPLE ALGORITHM FOR COMPUTING DETOUR INDEX OF NANOCLUSTERS

Let G be the chemical graph of a molecule. The matrix D = [dij ] is called the detour matrix of G, if dij is the length of longest path between atoms i and j. The sum of all entries above the main diagonal of D is called the detour index of G. In this paper, a new algorithm for computing the detour index of molecular graphs is presented. We apply our algorithm on copper and silver nanoclusters ...

An Asymptotically Optimal Heuristic for General Non- Stationary Finite-Horizon Restless Multi-Armed Multi- Action Bandits